Method and Apparatus for Modulating Haptic Feedback

ABSTRACT

The present invention concerns a method and apparatus for the modulation of an acoustic field for providing tactile sensations. A method of creating haptic feedback using ultrasound is provided. The method comprises the steps of generating a plurality of ultrasound waves with a common focal point using a phased array of ultrasound transducers, the common focal point being a haptic feedback point, and modulating the generation of the ultrasound waves using a waveform selected to produce little or no audible sound at the haptic feedback point.

FIELD OF THE INVENTION

The present invention concerns a method and apparatus for the modulationof an acoustic field for providing tactile sensations. Moreparticularly, but not exclusively, this invention concerns a method andapparatus for the modulation of an acoustic field for providing tactilesensations in order to provide an improved user experience.

BACKGROUND

Various interactive haptic technologies exist, which provide a user orusers with tactile information or feedback, often in combination withvisual information displayed on an interactive screen. For example,previous haptic feedback devices include pins moving to physicallychange a deformable surface. A pen connected to an articulated arm maybe provided, as in the SensAble PHANTOM device. Alternatively, a usermay wear, for example in the form of a glove, one or more actuatorswhich are activated to provide haptic feedback to a user. However, ineach of these technologies, a user requires physical contact with adeformable surface, a pen, or a specially adapted glove. Suchrequirements reduce the usability and spontaneity with which a user mayinteract with a system.

Tactile sensations on human skin can be created by using a phased arrayof ultrasound transducers to exert an acoustic radiation force on atarget in mid-air. Ultrasound waves are transmitted by the transducers,with the phase emitted by each transducer adjusted such that the wavesarrive concurrently at the target point in order to maximise theacoustic radiation force exerted.

Ultrasound haptic feedback systems create a vibrotactile sensation uponthe skin of a user of the system. The focused ultrasound creates enoughforce at the point of intersection to slightly displace the skin of auser. Typically, ultrasound haptic feedback systems use ultrasound witha frequency at or above 40 kHz, which is above the threshold forreceptors in the skin to feel. Therefore, a user can only detect theonset and cessation of such focused ultrasound. In order to provide asensation that is detectable by the receptors in skin, the focusedultrasound is modulated at a lower frequency, within the detectablerange of the receptors. This range is typically from 1 Hz to 500 Hz.

A side effect of the modulation is that the ultrasound breaks down andcreates a sound at the modulation frequency. Therefore, when creatingtactile feedback with a 200 Hz modulation frequency, a 200 Hz sound isalso produced. This audible sound may be annoying to users and is abarrier to ultrasound haptic technology being adopted.

The present invention seeks to mitigate the above-mentioned problems.Alternatively or additionally, the present invention seeks to provide animproved haptic feedback system.

SUMMARY OF THE INVENTION

The present invention provides, according to a first aspect, a method ofcreating haptic feedback using ultrasound comprising the steps of:

generating a plurality of ultrasound waves with a common focal pointusing a phased array of ultrasound transducers, the common focal pointbeing a haptic feedback point,

modulating the generation of the ultrasound waves using a waveformselected to produce little or no audible sound at the haptic feedbackpoint.

The method may comprise the step of generating a plurality of commonfocal points, each common focal point being a haptic feedback point.

The production of little or no audible sound at a haptic feedback pointis dependent on both the loudness of any sound produced, together withthe frequency at which any sound is produced. A small amount of audiblesound may be produced, and considered acceptable. The acceptability ofthe sound produced by the creation of haptic feedback may be dependenton the background noise audible at the haptic feedback point. In a noisyenvironment, the acceptable level of sound produced at the hapticfeedback point may be greater than the acceptable level of soundproduced in a quiet environment. Therefore, the intended use of thehaptic feedback system creating the haptic feedback and the environmentin which the system is located will determine the acceptable level ofsound production.

At high intensities, ultrasound becomes non-linear. This non-linearbehaviour allows the creation of haptic feedback but also causes theaudible sound produced in haptic feedback systems. An example of theeffect of the non-linearity of ultrasound waves is utilising the effectto create highly directive audible sound with parametric speakers. Thesound is produced due to the second derivative of the p² term of theWestervelt equation.

$\begin{matrix}{{{\nabla^{2}p} + {\frac{\delta}{c_{0}^{4}}\frac{\partial^{3}p}{\partial t^{3}}} + {\frac{\beta}{\rho_{0}c_{0}^{4}}\frac{\partial^{2}p^{2}}{\partial t^{2}}} - {\frac{1}{c_{0}^{2}}\frac{\partial^{2}p}{\partial t^{2}}}} = 0} & \left( {{Eq}.1} \right)\end{matrix}$

Where p is the sound pressure, which in the case of the invention is thedifference between the instantaneous sound pressure at a point and theambient sound pressure.

In the case of existing haptic feedback systems, a modulated phasedarray produces ultrasound modulated with a simple square wave pattern,i.e. the array is switched on and off at the modulation frequency.Therefore, the p² term is generating an approximate square wave at thefocal point of the phased array with a frequency that matches themodulation frequency. The square wave generated by the non-linearbreakdown of the ultrasound will create a relatively loud noise and isoff putting to a user of such a haptic feedback system.

In order to reduce or eliminate the audible noise, the applicant hasrealised it is necessary to avoid sharp changes in sound pressure levelthroughout the acoustic field. These sharp changes in pressure, asdemonstrated by the square wave example above, are turned intooscillations by the non-linear medium. By smoothing out the changes insound pressure level, the sound produced may be reduced to a lowerand/or inaudible level. The sound pressure may be continuously changed.The sound pressure may be continuously changed with the first derivativeof the rate of change also being continuous. The sound pressure may becontinuously changed with the second derivative of the rate of changebeing continuous. The maximum rate of change of sound pressure may bedependent on a number of factors, including the variability of thresholdhearing of humans with frequency of sound waves, and factors such as thetemperature and humidity of air in which the wave is being produced. Thegeneration of haptic feedback may be “tuned” in order that any soundproduced is suitably low for the intended use of the haptic feedbacksystem.

Alternatively or additionally, the sound produced by the method may lastonly a very short time. As humans do not perceive sounds that last onlyvery briefly, this can effectively make the sound produced inaudible.

One possible way of reducing the sound produced may be to avoid turningthe transducers off, and therefore avoid the emission of acoustic energyrapidly switching between 0% and 100% as in a square wave modulation.The method may comprise the step of varying the position of the commonfocal point. The position of the common focal point may be constantlyvaried. The position of the common focal point may oscillate about acentral focal point. For example, the phase delays of a phased arraycould be altered to defocus and refocus ultrasound to a feedback pointat the modulation frequency. The sound pressure level emitted by anindividual transducer in a phased array is small compared to the soundpressure level at the focal point, and so there will still be a largechange in sound pressure at the focal point. Therefore, this solutionmay have a relatively small effect.

The method may comprise the step of avoiding sharp pressure changes atthe focal point. The modulation may comprise selecting a waveform thatis an interpolation of the transducer phases and amplitudes. Thedetailed description shows various interpolated waveforms and thewaveforms that are produced at the focal point by the non-linearbreakdown of the ultrasound. The waveforms may be interpolated between afully on and fully off state. The interpolation curves may begeneralised between any two transducer phase and amplitudeconfigurations. The interpolation may be a linear interpolation. Theinterpolation may be a polynomial or trigonometric interpolation, suchas a cosine interpolation. The interpolation may be a parametric speakerinterpolation, arranged to result in a sinusoidal waveform beinggenerated at the focal point. The parametric speaker interpolation may,for example, be according to the same equation as that used to encode asinusoid into a parametric speaker beam to remove distortion. An exampleof such an equation can be found in Pompei (2002) “Sound fromUltrasound: The Parametric Array as an Audible Sound Source”, Ph.D. MIT:US, Eq 3.9. The interpolated waveform may produce smoother waveforms atthe focal point than the square wave modulation of the prior art.

The invention provides, according to a second aspect, a haptic feedbacksystem comprising:

a phased array comprising a plurality of transducers arranged to emitultrasound to create a haptic feedback point,

the phased array arranged to emit ultrasound according to a modulationwaveform having a shape that produces little or no sound when theultrasound converges at the haptic feedback point.

The haptic feedback system may comprise a control unit. The hapticfeedback system may comprise a driving unit. The driving unit may bearranged to drive the transducer to produce ultrasound. The control unitmay be arranged to send control signals to the driving unit. The controlunit may include a memory. The control unit may be arranged to modulatethe output of the transducer according to a particular modulationwaveform. The modulation waveform may be linear. The modulation waveformmay be a polynomial or trigonometric interpolation, for example, acosine interpolation. The modulation waveform may correspond to aparabolic speaker interpolation. The control unit may be a PC or othersuitable computer device.

According to a third aspect, the invention provides a computer programproduct, the computer program comprising a series of instructions, theseries of instructions such that when run on a control unit associatedwith a haptic feedback system according to the second aspect of theinvention, the haptic feedback system operates such that the methodsteps according to the first aspect of the invention are carried out.

It will of course be appreciated that features described in relation toone aspect of the present invention may be incorporated into otheraspects of the present invention. For example, the method of theinvention may incorporate any of the features described with referenceto the apparatus of the invention and vice versa.

DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described by way ofexample only with reference to the accompanying schematic drawings ofwhich:

FIG. 1 shows a schematic view of a haptic feedback system according to afirst embodiment of the invention;

FIG. 2 shows a prior art square wave modulation pattern and theresultant waveform produced at the focal point;

FIG. 3 shows a linear interpolation modulation pattern and the resultantwaveform produced at the focal point according to a second aspect of theinvention;

FIG. 4 shows a cosine interpolation modulation pattern and the resultantwaveform produced at the focal point according to a third aspect of theinvention;

FIG. 5 shows a parametric speaker interpolation modulation pattern andthe resultant waveform produced at the focal point according to a fourthaspect of the invention;

FIG. 6 shows an acoustic field generated at a focal point by a cosineinterpolation modulation;

FIG. 7 shows an acoustic field generated at a focal point by aparametric speaker interpolation modulation; and

FIG. 8 shows an acoustic field generated at a focal point by a squarewave modulation.

DETAILED DESCRIPTION

In an example embodiment of the method, firstly the 3D position of afocal point is decided. A phased array is arranged to create an acousticfield, with the phases and amplitudes of each transducer calculated toachieve a high pressure at the focal point and a low pressure insurrounding areas. Two states then exist, firstly the focal point state,with the computed phases and amplitudes, and secondly the off state,with all of the transducers of a phased array set at zero amplitude. Afrequency at which to modulate the feedback is then chosen in dependenceon the desired feel of the feedback. Then a modulation waveform ischosen at the desired frequency, the modulation frequency chosen tominimise or reduce the audible sound produced at the focal point. Anexample modulation waveform is a cosine waveform. The modulationwaveform is then applied to the operation of the transducers tointerpolate between the two states identified above.

A more specific example, as applied to a particular haptic feedbacksystem, is now described with reference to FIG. 1 .

FIG. 1 shows an example haptic feedback system 10 comprising atransducer array 12, a screen 14, a projector 16, a hand tracker 20, aPC 22, a driving unit 24, and a user's hand 26. The system 10 is shownto illustrate the invention, which is in no way limited to a particularsystem for producing haptic feedback using ultrasound. The transducerarray 12 is located underneath the screen 14 and arranged such thatpressure patterns may be transmitted through the screen 14 to a regionabove the screen 14. In this particular embodiment, the transducer arraycomprises 320 muRata MA40S4S transducers arranged in a 16×20 gridformation. Each transducer unit is 10 mm in diameter and the transducersare positioned with no gap between them in order to minimise thetransducer array 12 footprint. The transducers produce a large amount ofsound pressure (20 Pascals of pressure at a distance of 30 cm) and havea wide angle of directivity (60 degrees). The transducers are arrangedto transmit ultrasound waves at a frequency of 40 kHz. The projector 16is arranged to project visual information onto the screen 14 from abovethe screen 14 as shown. In an alternative embodiment, the projector maybe placed between the transducer array and the screen, with theprojection coming from below the screen.

A user interacts with this visual information and the movement andposition of the user's hand 26 is tracked by the hand tracker 20. Inthis particular embodiment, the hand tracker 20 is a Leap Motioncontroller arranged to provide the 3D coordinates of the user'sfingertips and palm at up to 200 frames per second. The system 10 iscontrolled by a PC 22, which sends control data to the projector 16,receives user data from the hand tracker 20, and controls the drive unit24 for driving the transducer array 12. The PC 22 controls the drivingunit 24 such that a pressure pattern is created in the region above thetransducer array 12. In response to the hand movements of a user, the PC22 may drive the driving unit 24 to cause the transducer array 12 tochange the pressure pattern formed above the transducer array 12.

In order to compute the amplitude and phase of the acoustic wave eachacoustic transducer must transmit for the desired pressure pattern to becreated, an algorithm adapted from that proposed by Gavrilov (“Thepossibility of generating focal regions of complex configurations inapplication to the problems of stimulation of human receptor structuresby focused ultrasound”, L. R. Gavrilov, 2008, Acoustical Physics Volume54, Issue 2, pp 269-278, Print ISSN 1063-7710) may be used. A volumetricbox is defined above the transducer array 12. Within the box, aplurality of control points are defined. The control points mayrepresent points where a maximum pressure value is desired, or pointswhere minimum pressure values are desired. The pressure values aremaximised or minimised by maximising or minimising the intensity of theultrasound emitted by the transducer array 12 which is incident at thecontrol points.

An algorithm is used to model the outputs of each of the transducers inthe transducer array 12 required to obtain each of the desired pressurepatterns which may be created within the volume defined above thetransducer array 12. The algorithm may be split into three steps.

Firstly, the acoustic field generated by a single transducer iscalculated to create a large modelled volume. Thereby, the phase andamplitude at any point within the modelled volume may be determined byoffsetting the sample transducer for the position, phase, and amplitude,of each of the transducers in the real transducer array, and combiningthese values.

Secondly, the control points are defined in the 3D volume above thetransducer array such that the control points take on the requireddistribution. The control points may be points of maximum intensity orminimum intensity (also known as null points). In addition to a 3Dlocation, the desired modulation frequency of the maximum control pointsmay be specified. Thirdly, the optimal phases are calculated using aminimum norm solver so that the resulting acoustic field is as close aspossible to that specified by the control points. There may be more thanone solution that will create an optimal focusing to the control points,but some solutions create a higher intensity than others. Solutions aretherefore iteratively generated to find the one that creates the highestintensity.

The method according to an aspect of the invention comprises obtaining amodulation frequency that produces the required tactile sensation. Forexample, a relatively slow modulation frequency of 16 Hz would provide aslow, pulsing, sensation. A higher modulation frequency of 200 Hz wouldproduce a near-continuous feeling. A modulation waveform is thenselected at that frequency, which produces little or no audible sound atthe feedback point. The modulation waveform may comprise aninterpolation based on the required phase and amplitude of the waveformcalculated as described above.

FIGS. 2 to 6 show a graph on the left hand side which represents themodulation waveform applied to the ultrasound emitted by an ultrasoundtransducer. The graph on the right hand side of the figures representsthe audible waveform created at the focal point of the ultrasoundtransducer. Generally, the greater the amplitude and the more jagged thefeedback waves created at the focal point, the louder the sound beingproduced will be.

In prior art systems, the modulation of the ultrasound corresponds to asimple square wave pattern, as shown in the graph on the left hand sideof FIG. 2 , where the array of transducers is simply turned on and offat the modulation frequency. The graph on the right hand side of FIG. 2shows the waveform produced at the focal point of the ultrasoundtransducer when using a square wave modulation pattern. As is clear, thewaveform is far from smooth and also the amplitude of the waveform isrelatively high. This will result in a potentially loud and irritatingsound being produced at the focal point of the haptic feedback system.

FIG. 3 shows an alternative modulation waveform, where the ultrasound isvaried according to a linear interpolation. As can be seen in the graphon the right hand side of FIG. 3 , the waveform produced at the focalpoint is smoother than that shown in FIG. 2 , with an amplitude which issignificantly smaller. Therefore, the sound produced at the focal pointwill be reduced compared to a square wave modulation.

FIG. 4 shows an alternative modulation waveform, where the ultrasound isvaried according to a cosine interpolation. As can be seen in the graphon the right hand side of FIG. 4 , the waveform produced at the focalpoint is smoother than that shown in FIG. 2 , with an amplitude which issignificantly smaller. Therefore, the sound produced at the focal pointwill be reduced compared to a square wave modulation.

FIG. 5 shows an alternative modulation waveform, where the ultrasound isvaried according to a parametric speaker interpolation. As can be seenin the graph on the right hand side of FIG. 5 , the waveform produced atthe focal point is smoother than that shown in FIG. 2 , with anamplitude which is significantly smaller. Therefore, the sound producedat the focal point will be reduced compared to the square wavemodulation.

FIGS. 6, 7, and 8 , show the acoustic field of audible waveforms that isproduced from different modulation waveforms when a focal point iscreated from five point sources. The waveform at various pointsthroughout the field are highlighted for comparison. FIG. 6 represents acosine interpolation, FIG. 7 represents a parametric speakerinterpolation, and FIG. 8 represents a square wave modulation method. Ascan be seen, FIG. 6 shows the smoothest, most uniform field. FIG. 7shows a field which is not as smooth and uniform as FIG. 6 , thoughstill considerably smoother and more uniform than that shown in FIG. 8 .Therefore it is evident that the cosine interpolation provides theoptimum modulation compared to the others discussed. On investigation,the skilled person may discover alternative modulation waveforms whichperform as well as or better than a cosine interpolation, whilst stillfalling within the scope of the present invention.

Whilst the present invention has been described and illustrated withreference to particular embodiments, it will be appreciated by those ofordinary skill in the art that the invention lends itself to manydifferent variations not specifically illustrated herein.

Where in the foregoing description, integers or elements are mentionedwhich have known, obvious or foreseeable equivalents, then suchequivalents are herein incorporated as if individually set forth.Reference should be made to the claims for determining the true scope ofthe present invention, which should be construed so as to encompass anysuch equivalents. It will also be appreciated by the reader thatintegers or features of the invention that are described as preferable,advantageous, convenient or the like are optional and do not limit thescope of the independent claims. Moreover, it is to be understood thatsuch optional integers or features, whilst of possible benefit in someembodiments of the invention, may not be desirable, and may therefore beabsent, in other embodiments.

1-24. (canceled)
 25. A method of creating haptic feedback usingultrasound comprising the steps of: using a plurality of ultrasonictransducers, generating a plurality of ultrasound waves, wherein atleast two of the plurality of ultrasound waves form a focal point,wherein the focal point is a haptic feedback point; first modulating thegenerating of the plurality of ultrasound waves using a waveform toproduce a haptic sensation at the haptic feedback point, which producesless audible sound pressure level than would be produced at the hapticfeedback point when second modulating the generating of the plurality ofultrasound waves using a square wave modulation pattern.
 26. The methodas in claim 25, wherein the waveform is varied according to a linearinterpolation.
 27. The method as in claim 25, wherein the waveform isvaried according to a cosine interpolation.
 28. The method as in claim25, wherein the waveform is varied according to a polynomialinterpolation.
 29. The method as in claim 25, wherein the waveform isvaried according to trigonometric interpolation.
 30. The method as inclaim 25, wherein the waveform is varied according to a parametricspeaker interpolation.
 31. The method as in claim 25, wherein the squarewave modulation pattern matches the modulation frequency of thewaveform.
 32. The method as in claim 31, wherein, at the focal point, afirst peak amplitude of the first modulating is equal to a second peakamplitude of the second modulating.
 33. The method as in claim 25,wherein the waveform comprises an interpolation of phase of at least oneof the plurality of the ultrasonic transducers.
 34. The method as inclaim 25, wherein the waveform comprises an interpolation of amplitudeof at least one of the plurality of the ultrasonic transducers.
 35. Themethod of claim 25, wherein the generating a plurality of ultrasoundwaves use ultrasound with a frequency at or above 40 kHz.
 36. The methodof claim 25, wherein the first modulating the generating of theplurality of ultrasound waves uses a frequency from 1 Hz to 500 Hz. 37.The method of claim 25, further comprising: varying a position of thefocal point.
 38. The method of claim 37, wherein the position of thefocal point is constantly varied.
 39. The method of claim 37, whereinthe position of the focal point oscillates about a central point.
 40. Asystem comprising: a plurality of ultrasonic transducers for generationof a plurality of ultrasound waves, wherein at least two of theplurality of ultrasound waves form a focal point, wherein the focalpoint is a haptic feedback point; a first modulation of the generationof the plurality of ultrasound waves using a waveform to produce ahaptic sensation at the haptic feedback point, which produces lessaudible sound pressure level than would be produced at the hapticfeedback point during a second modulation of the generation of theplurality of ultrasound waves using a square wave modulation pattern.41. The system as in claim 40, wherein the waveform is varied accordingto a linear interpolation.
 42. The system as in claim 40, wherein thewaveform is varied according to a cosine interpolation.
 43. The systemas in claim 40, wherein the waveform is varied according to a polynomialinterpolation.
 44. The system as in claim 40, wherein the waveform isvaried according to trigonometric interpolation.
 45. The system as inclaim 40, wherein the waveform is varied according to a parametricspeaker interpolation.
 46. The system as in claim 40, wherein the squarewave modulation pattern matches the modulation frequency of thewaveform.
 47. The system as in claim 46, wherein, at the focal point, apeak amplitude of the first modulation is equal to a peak amplitude ofthe second modulation.
 48. The system as in claim 40, wherein thewaveform comprises an interpolation of phase of at least one of theplurality of the ultrasonic transducers.
 49. The system as in claim 40,wherein the waveform comprises an interpolation of amplitude of at leastone of the plurality of the ultrasonic transducers.
 50. The system ofclaim 40, wherein the generation of the plurality of ultrasound wavesuse ultrasound with a frequency at or above 40 kHz.
 51. The system ofclaim 40, wherein the first modulation of the generation of theplurality of ultrasound waves uses a frequency from 1 Hz to 500 Hz. 52.The system of claim 40, further comprising: variance of a position ofthe focal point.
 53. The system of claim 52, wherein the position of thefocal point is constantly varied.
 54. The system of claim 52, whereinthe position of the focal point oscillates about a central point.